Use of Instrumental Variable Analyses for Evaluating Comparative Effectiveness in Empirical Applications of Oncology: A Systematic Review.

PURPOSE: This systematic review aims to characterize the use and trends of instrumental variables (IVs) in oncology research, assess the quality and completeness of IV reporting, and evaluate the agreement and interpretation of IV results in comparison with other techniques used for determining comparative effectiveness in observational research. METHODS: We performed a systematic search of observational empirical oncology papers evaluating the comparative effectiveness of cancer treatments using IV methods. EMBASE and MEDLINE (through June 2021) were used for a keyword search; Scopus and Web of Science were used for a citation search. Publication details and characteristics of IV analysis and reporting were extracted from each study to examine the uptake and quality of IV applications. RESULTS: Sixty-five empirical papers were identified from February 2001 through June 2021. Geographic variation (50.8%) was the most common type of IV used, and the majority of IV applications constructed binary instruments (53.8%). Concurrent analyses using another non-IV method to adjust for confounding were conducted in 56 (86.2%) studies, 17 (30.4%) of which produced results divergent from IV approaches. We observed a modest uptake of IV methods between 2011 and 2021 together with its dissemination, which remained fairly limited to the United States (76.9%). The quality and completeness of IV reporting varied greatly. The underlying assumptions required for a valid IV analysis were only accounted for in full by 20 (30.8%) studies. CONCLUSION: There are limited use and variable quality of IV analyses in oncology. Future research should look to establish standards to better facilitate the quality, transparency, and completeness of IV reporting in this setting.

Do Early Phase Oncology Trials Predict Clinical Efficacy in Subsequent Biomarker-Enriched Phase III Randomized Trials?

BACKGROUND: Promising early phase trial results of biomarker-targeted therapies have occasionally led to regulatory approval. OBJECTIVE: We examined if early phase trials were predictive of efficacy in randomized controlled trials (RCTs) with matching treatment settings. PATIENTS AND METHODS: Cancer drug RCTs conducted between January 2006 and March 2021 were identified through Clinicaltrials.gov. Biomarker-enriched RCTs and associated matching early phase trials were included. Trial pairs were compared using objective response rate (ORR) and progression-free survival (PFS). We examined whether early phase trials results were associated with RCT results using logistic regression. RESULTS: The search yielded 2157 unique RCTs and 27 RCTs pairing with early phase trials were included. Based on average difference of trial pairs, ORR was similar (1.6%; 95% confidence interval (CI) - 2.5 to 5.6, p = 0.50) and median PFS was higher in early phase trials (2.0 months; 95% CI 0.9-3.0, p < 0.05). On an individual pair basis, there was large variability in difference for ORR (range - 23.9 to 20.2%) and median PFS (range - 0.8 to 7.4 months). The probability of the RCT meeting its primary endpoint is 95% (95% prediction interval (PI) 72.8-99.3%) when the early phase trial ORR is 77.7%. CONCLUSIONS: Overall, in early phase trials, ORR has minimal bias and median PFS appears to be slightly overestimated. Substantial variability between trials suggests early phase trial results may be inconsistent with subsequent RCT. Early phase trial results may be associated with RCTs meeting their primary endpoint when ORR is very high; however, caution must be exercised when using early phase trials as representative of RCTs.

Examining the association between oncology drug clinical benefit and the time to public reimbursement.

BACKGROUND: We examined if oncology drug indications with high clinical benefit, as measured by the American Society of Clinical Oncology Value Framework (ASCO-VF) and European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS), received public reimbursement status faster than those with lower clinical benefit from the time of pan-Canadian Oncology Drug Review (pCODR) recommendation. METHODS: Oncology drug indications submitted to pCODR between July 2011 and October 2018 were examined. Included indications had a regulatory approval date, completed the pCODR review process, received a positive pCODR recommendation, and been funded by at least one province. Trials cited for clinical efficacy were used to determine the clinical benefit (per ASCO-VF and ESMO-MCBS) of drug indications. RESULTS: Eighty-four indications were identified, yielding 65 ASCO-VF and 50 ESMO-MCBS scores. The mean ASCO-VF and ESMO-MCBS scores were 44.9 (SD = 21.1) and 3.3 (SD = 1.0), respectively. The mean time to provincial reimbursement from pCODR recommendation was 13.2 months (SD = 9.3 months). Higher ASCO-VF and ESMO-MCBS scores had low correlation with shorter time to reimbursement, (ρ = -0.21) and (ρ = 0.24), respectively. In the multivariable analyses, ASCO-VF (p = 0.40) and ESMO-MCBS (p = 0.31) scores were not significantly associated with time to reimbursement. Province and year of pCODR recommendation were associated with time to reimbursement in both ASCO and ESMO models. CONCLUSIONS: Oncology drug indications with higher clinical benefit do not appear to be reimbursed faster than those with low clinical benefit. This suggests the need to prioritize oncology drug indications based on clinical benefit to ensure quicker access to oncology drugs with the greatest benefits.

Assessing the benefit of cancer drugs approved by the European Medicines Agency using the European Society for Medical Oncology Magnitude of Clinical Benefit Scale over time.

BACKGROUND: Increasingly, cancer drugs are being approved based on surrogate measurements of efficacy. Clinically meaningful data, such as overall survival (OS) and quality of life, are often only presented in subsequent publications. We examined if the clinical benefit of cancer drugs, as measured by the European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS), improves post-European Medicines Agency (EMA) approval as more data emerges. METHODS: Cancer drug indications approved by the EMA from January 2006 to December 2016 were reviewed and trials cited for efficacy were identified. Primary and subsequent publications (up to December 2019) of scorable trials were included. Changes in benefit over time were measured using ESMO-MCBS thresholds for non-curative (≥4 for substantial, =3 for intermediate and ≤2 for low benefit) and curative intent (A or B for major benefit) scoring. RESULTS: Fifty-five non-curative and two curative intent trials were included. At approval, 29.1% of non-curative trials were substantial, 45.5% intermediate and 25.5% low benefit. For curative intent trials, one displayed major benefit and one displayed no major benefit. We identified 82 subsequent publications for reassessment. A change in ESMO-MCBS classification was seen in 34.5% of non-curative trials (11 raised and 8 lowered). At 3-year reassessment, 36.4% of non-curative trials were substantial, 34.5% intermediate and 29.1% low benefit. Both curative trials showed no major benefit at reassessment. CONCLUSION: As over a third of trials changed classification, in either direction, reassessing the ESMO-MCBS score of approved cancer drugs may help to inform patients and ensure ongoing relevance of regulatory and reimbursement decisions.